{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Test sortgs.py on Colab.ipynb",
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/WittmannF/sort-google-scholar/blob/master/Test_sortgs_py_on_Colab.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M36VFanosbkb",
        "colab_type": "text"
      },
      "source": [
        "## Try `sortgs.py` on Google Colab\n",
        "This is a jupyter envirnment where you can try the code of the repository without installing anything. The only limitation is the robot checking problem which would require selenium and manual solution of the captchas, but for trying a few keywords, it should work! \n",
        "\n",
        "> **INSTRUCTIONS:** If this is the first time you are using a jupyter environment, you simply have to run the code blocks using the keyword `SHIFT` + `ENTER`. Make sure to update the keyword parameters when required. \n",
        "\n",
        "First, let's clone the repository:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oPot8aWcsfei",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!git clone https://github.com/WittmannF/sort-google-scholar.git"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pM5XMRmVspR7",
        "colab_type": "text"
      },
      "source": [
        "Now, let's open the folder from the cloned repository:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LQ0mCxVNrlCB",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "4501af7d-39d8-497d-af7b-2632f2fc856b"
      },
      "source": [
        "cd sort-google-scholar/"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/sort-google-scholar\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KALGyCEJsvTL",
        "colab_type": "text"
      },
      "source": [
        "Next, let's run the code. Make sure to change \"deep learning\" to the keyword that you would like to have it ranked."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dohZcNUmrost",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 504
        },
        "outputId": "6c61226a-8ab8-4609-8fac-5127c4e48d9f"
      },
      "source": [
        "# Change 'deep learning' with the keyword that you would like to check\n",
        "!python sortgs.py --kw \"deep learning\" --sortby \"cit/year\""
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Loading next 10 results\n",
            "sortgs.py:239: UserWarning: Year not found for Could not catch title, appending 0\n",
            "  warnings.warn(\"Year not found for {}, appending 0\".format(title[-1]))\n",
            "Loading next 20 results\n",
            "Loading next 30 results\n",
            "Loading next 40 results\n",
            "Loading next 50 results\n",
            "Loading next 60 results\n",
            "Loading next 70 results\n",
            "Loading next 80 results\n",
            "Loading next 90 results\n",
            "Loading next 100 results\n",
            "Column name to be sorted not found. Sorting by the number of citations...\n",
            "'cit/year'\n",
            "                                                 Author  ... cit/year\n",
            "Rank                                                     ...         \n",
            "1                             LeCun, Y Bengio, G Hinton  ...     3846\n",
            "3                     Goodfellow, Y Bengio, A Courville  ...     2616\n",
            "9                                           Schmidhuber  ...     1360\n",
            "4              Mnih, K Kavukcuoglu, D Silver, A Graves…  ...      421\n",
            "41    Litjens, T Kooi, BE Bejnordi, AAA Setio, F Cio...  ...      689\n",
            "...                                                 ...  ...      ...\n",
            "34                                     Author not found  ...        0\n",
            "23                                     Author not found  ...        0\n",
            "12                                     Author not found  ...        0\n",
            "5                                      Author not found  ...        0\n",
            "111                                    Author not found  ...        0\n",
            "\n",
            "[111 rows x 6 columns]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "s_nuxpy_s_9c",
        "colab_type": "text"
      },
      "source": [
        "> _**NOTE:** It is normal to get some warnings, for example year not found or author not found. However, if you get the robot checking warning, then it might not work anymore in the IP that you have on Google Colab. You can try going in 'Runtime' and 'Reset all runtimes' to get a new IP, but it is not garanteed that it will work. If the problem persists, then you will have to run locally using selenium and solve the captchas manually. Make sure to avoid running this code too often to avoid the robot checking problem._\n",
        "\n",
        "Next, you will see that a csv file with the name of the keyword was created (which is `deep_learning.csv` in my case). "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "94792EoBrrBU",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "c8e256c2-b7df-4c93-94f2-14d71a7f8050"
      },
      "source": [
        "ls"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "deep_learning.csv  \u001b[0m\u001b[01;34mexamples\u001b[0m/  README.md  \u001b[01;34msort-google-scholar-V1\u001b[0m/  sortgs.py\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QQIb9oYou9GM",
        "colab_type": "text"
      },
      "source": [
        "Let's import this file to visualize some results (make sure to change `deep_learning.csv` to the file name that was created in your case):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NVoHI4zTr5pB",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import pandas as pd\n",
        "df = pd.read_csv('deep_learning.csv') # OBS: Update the file name"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "S1NzeUq0sOu-",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 578
        },
        "outputId": "29bdad8a-17e1-415d-a2e3-827cfe71889a"
      },
      "source": [
        "df.head(10)"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Rank</th>\n",
              "      <th>Author</th>\n",
              "      <th>Title</th>\n",
              "      <th>Citations</th>\n",
              "      <th>Year</th>\n",
              "      <th>Source</th>\n",
              "      <th>cit/year</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>LeCun, Y Bengio, G Hinton</td>\n",
              "      <td>Deep learning</td>\n",
              "      <td>19232</td>\n",
              "      <td>2015</td>\n",
              "      <td>https://www.nature.com/articles/nature14539</td>\n",
              "      <td>3846</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>3</td>\n",
              "      <td>Goodfellow, Y Bengio, A Courville</td>\n",
              "      <td>Deep learning</td>\n",
              "      <td>10466</td>\n",
              "      <td>2016</td>\n",
              "      <td>https://books.google.com/books?hl=en&amp;lr=&amp;id=om...</td>\n",
              "      <td>2616</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>9</td>\n",
              "      <td>Schmidhuber</td>\n",
              "      <td>Deep learning in neural networks: An overview</td>\n",
              "      <td>6799</td>\n",
              "      <td>2015</td>\n",
              "      <td>https://www.sciencedirect.com/science/article/...</td>\n",
              "      <td>1360</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>Mnih, K Kavukcuoglu, D Silver, A Graves…</td>\n",
              "      <td>Playing atari with deep reinforcement learning</td>\n",
              "      <td>2949</td>\n",
              "      <td>2013</td>\n",
              "      <td>https://arxiv.org/abs/1312.5602</td>\n",
              "      <td>421</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>41</td>\n",
              "      <td>Litjens, T Kooi, BE Bejnordi, AAA Setio, F Cio...</td>\n",
              "      <td>A survey on deep learning in medical image ana...</td>\n",
              "      <td>2067</td>\n",
              "      <td>2017</td>\n",
              "      <td>https://www.sciencedirect.com/science/article/...</td>\n",
              "      <td>689</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>24</td>\n",
              "      <td>Sutskever, J Martens, G Dahl, G Hinton</td>\n",
              "      <td>On the importance of initialization and moment...</td>\n",
              "      <td>2031</td>\n",
              "      <td>2013</td>\n",
              "      <td>http://www.jmlr.org/proceedings/papers/v28/sut...</td>\n",
              "      <td>290</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>6</td>\n",
              "      <td>Deng, D Yu</td>\n",
              "      <td>Deep learning: methods and applications</td>\n",
              "      <td>1884</td>\n",
              "      <td>2014</td>\n",
              "      <td>http://www.nowpublishers.com/article/Details/S...</td>\n",
              "      <td>314</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>27</td>\n",
              "      <td>Erhan, Y Bengio, A Courville, PA Manzagol…</td>\n",
              "      <td>Why does unsupervised pre-training help deep l...</td>\n",
              "      <td>1857</td>\n",
              "      <td>2010</td>\n",
              "      <td>http://www.jmlr.org/papers/v11/erhan10a.html</td>\n",
              "      <td>186</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>2</td>\n",
              "      <td>Ngiam, A Khosla, M Kim, J Nam, H Lee…</td>\n",
              "      <td>Multimodal deep learning</td>\n",
              "      <td>1785</td>\n",
              "      <td>2011</td>\n",
              "      <td>http://ai.stanford.edu/~ang/papers/icml11-Mult...</td>\n",
              "      <td>198</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>20</td>\n",
              "      <td>Dong, CC Loy, K He, X Tang</td>\n",
              "      <td>Learning a deep convolutional network for imag...</td>\n",
              "      <td>1734</td>\n",
              "      <td>2014</td>\n",
              "      <td>https://link.springer.com/chapter/10.1007/978-...</td>\n",
              "      <td>289</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   Rank  ... cit/year\n",
              "0     1  ...     3846\n",
              "1     3  ...     2616\n",
              "2     9  ...     1360\n",
              "3     4  ...      421\n",
              "4    41  ...      689\n",
              "5    24  ...      290\n",
              "6     6  ...      314\n",
              "7    27  ...      186\n",
              "8     2  ...      198\n",
              "9    20  ...      289\n",
              "\n",
              "[10 rows x 7 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "t7ZCDWaksZxj",
        "colab_type": "text"
      },
      "source": [
        "Above we have the top 10 papers of deep learning ranked by citations per year. It is incredible that some papers are very far away from the original rank. For example, the fourth most cited paper per year is originally the number 41 in the original rank of Google! "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-xx5GEHuybPd",
        "colab_type": "text"
      },
      "source": [
        "Finally, let's download the CSV results (make sure to update the CSV name here as well):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Svc1GCM2yaLP",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from google.colab import files\n",
        "files.download('deep_learning.csv') # Update the file name here"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "09OFwDdM2K5h",
        "colab_type": "text"
      },
      "source": [
        "OBS: If you get an error running the previous line, you can download the file manually by clicking in the left pane icon (below the `+ Code` button), then click in files, expand the `sort-google-scholar` folder, right click and download in the csv file:\n",
        "\n",
        "![](https://i.stack.imgur.com/KKJXFm.png)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pM_Bb4MH14eI",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}